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Abstract 



We consider the recursive equation "a;(n + 1) = A{n) (8) x{n)" where x{n + 1) and x{n) are 
M'^-valued vectors and A{n) is an irreducible random matrix of size k x k. The matrix-vector 
multipheation in the (max,+) algebra is defined by {A{n) ® .x(n))j = maxj{Aij{n) + Xj{n)). 
This type of equation can be used to represent the evolution of Stochastic Event Graphs which 
include cyclic Jackson Networks, some manufacturing models and models with general blocking 
(such as Kanban). Let us assume that the sequence {A{n), n G N} is i.i.d or more generally 
stationary and ergodie. The main result of the paper states that the system couples in finite 
time with a unique stationary regime if and only if there exists a set of matrices C such that 
P{A{0) G C} > and the matrices C G C have a unique periodic regime. 



1 Introduction 



Let us consider the following recursive equation: 



Xi{n + 1) 



maxj(Ajj(n) + Xj{n)) 



(1) 



The sequences {Aij{n)} are given (exogenous data). The process we want to study is the 
sequence of vectors {x(n) = (xi(n), . . . , Xi{n), . . . )'}. The vector xq is the initial condition. 
Because of the generality of Equation ([T]), it appears in many different types of applications. 
In fact, this equation appears in statistical mechanics in the study of crystal structures, see 
Griffiths [26]. It is also common under one form or another in economic and control literature, 
see Yakovenko & Kontorer [S^. In fact, it is the basic Bellman equation of dynamic optimization 
in discrete time, for a finite state space. Recently, this kind of equation has become very popular 
in the study of Discrete Events Dynamic Systems (DEDS), see for example the recent textbooks 
of Baccelli, Cohen, Olsder & Quadrat [Ij and Glasserman & Yao [23]. Let us insist on the last 
point. 

A large class of computer or communication networks accepts a description as DEDS. Different 
approaches have been proposed to model DEDS. Petri Nets is one of the most common formal- 
ism. More precisely, a sub-class of Petri Nets, Event Graphs, appears to be very efficient in 
describing models with synchronization, blocking and/or fork-join properties. We can mention 
Job-Shop models (see Cohen, Dubois, Quadrat & Viot [16] or [4]), cyclic Jackson Networks (see 
Section 12. 2p or asymmetric exclusion models as examples. On the other hand. Event Graphs 
cannot be used to model systems with routings. 

We can describe the evolution of an Event Graph by the daters associated with the transitions 
(nodes) of the graph. It is well known, see for example @], that the evolution of the daters of 
an Event Graph can be represented in the form of Equation ([T|). 

Generalized Semi-Markov Processes (GSMP) are another common formalism for DEDS. It is 
shown in [23] that a GSMP with convex and homogeneous structural properties admits a rep- 
resentation of the form In order to give a flavor of the modeling power of Equation ([T|) and 
in order to motivate the practical interest of this work, we will present two different models in 
Section [2l 

• The first model appears in the modeling and the analysis of parallel programs and 
architectures. It is a task graph with 'and' synchronizations also known as PERT Network. 

• The second model is a closed cyclic Jackson Network. We will consider both infinite 
and finite buffers with various blocking modes. 

It is very fruitful to use a matrix- vector notation for Equation ([T|). We define the following 
"(max,+)" notations: 



We define also the k x k matrix A(n) = {Aij{n),i,j = l,...,k} and the column vector 
x{n) = (xi(n),... ,Xk{n)y. With these notations, the basic Equation ^ takes a very simple 
and convenient form. In fact, it can be rewritten as: 



Vx, y € M U { — oo}, X (B y = max(a;, y), x y = x + y . 



x{n + 1) = A{n) (g) x{n) . 



(2) 
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The matrix- vector product is defined in a natural way just by replacing + and x by © and 0, 

i.e. {A (g) x)i = Aij (g) Xj = maKj{Aij +Xj). 

Historically, the first approach has been to consider deterministic systems where Aij{n) = Aij. 
It is natural to consider a stochastic extension where {Aij{n)} is a sequence of random matrices. 
As a consequence, here is an equivalent way of introducing our subject: it is a counterpart of 
the classical theory of products of random matrices (see Furstenberg & Kesten ^20j or Bougerol 
& Lacroix [12]) but in another algebraic structure, the (max,-|-) algebra. 

For systems described by Equation (l2|), we will consider two kinds of asymptotic results. 

• First order limits, on ratios: 

hmM!^, hm^. (3) 

n n n n 

• Second order limits, on differences: 

lim Xi{n + 1) — Xi{n), Mi, lim x,(n) — Xi{n), \/i ^ j . (4) 

n n 

A first order limit is a cycle time or equivalently the inverse of a throughput. Second order limits 
are related to waiting and idle times, queue length and frequency of occupation. More insights 
on the relations between these limits and quantities of interest for the system will be provided 
in Section [2j Our goal is to find stationary regimes for second order limits. Multiple stationary 
regimes will mean multiple possible regimes for waiting times or queue lengths, depending on 
the initial condition. 

Among the systems modeled by Equation ([2]), we can distinguish two classes: the open (or non- 
autonomous) systems and the closed (or autonomous) ones. Open systems have been exhaus- 
tively treated by Baccelli [3] [3| (for both first and second order limits). Problems of existence 
and uniqueness of first order limits for closed systems have been solved by Cohen [T7] (see also 
[3]). These results are recalled in ^6.11 This paper deals with the open question of existence 
and uniqueness of second order limits for closed systems. These problems were considered in 
several earlier papers (Resing, de Vries, Hooghiemstra, Keane & Olsder [33] and [32], Baccelli 
[3]) but only sufficient conditions of uniqueness were known. The approach we use is new and 
exploits completely the common hidden algebraic structure of the different models we consider. 
It enables us to obtain necessary and sufficient conditions for stability (in some cases) together 
with simple proofs. 

The conditions we give are based on the structure of deterministic matrices chosen in the support 
of the random matrix A(0). One of the main results states that the system couples in finite 
time with a unique stationary regime if and only if there exists a set of matrices C such that 
P{A(0) G C} > and the matrices C € C have a unique periodic regime. The proof makes use 
of Borovkov's theory of renovating events, see Borovkov &: Foss |10j This theory appears 
to be much more tractable than classical Harris regeneration due to the specific form of our 
recursive equations. 

The paper is organized as follows. We introduce two models in Section[2l cyclic Jackson Networks 
and task graphs with random precedences. Sections [3l S] and [5] are presenting the tools that 
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we are using in the paper. They can be skipped by people knowing the subject. Section [3] is 
devoted to the (max,+) algebra, Section U] to the spectral theory in this algebra and Section [5] 
to Borovkov's theory of renovating events. Section [6] presents the main results. In 16.11 we recall 
some results from [3] and [T7]. In 16.21 we state some preliminary results. In 16.31 and 16.41 we give 
sufficient conditions for the stability of discrete and general models respectively. In Section (Tj 
we establish the converses of the results of the previous section. In Section [HI we weaken the 
assumptions under which some of our results apply. Finally, for convenience, some of the proofs 
are given in Appendix. 

2 Two Motivating Models 

2.1 Task graphs 

We consider a parallel program executed on several identical processors. We model it by its 
precedence graph r. If we consider a system of k processors, the graph r has a set of nodes 
which is /c X N. The node (i, n) represents the n-th task to be executed at processor i. The 
arcs between nodes represent the synchronization constraints. There is an arc between the node 
{i,n) and the node {j,m) (notation : (i,n) — > {j,m)) if the n-th task at processor i has to be 
completed in order for the m-th task at processor j to be enabled. The execution of a task 
begins as soon as all the tasks of its incoming arcs are completed. Each task has a duration 
which may depend on the processor. 

Let us consider a task graph with synchronizations only between consecutive levels n (i.e. nodes 
(1, n), . . . , (A:, n)) and n + 1. We assume that the synchronizations depend on n. We denote by 
C{i,n) the set of nodes j such that {i,n) — > (j, n + 1). We suppose that for all i, there exists a 
probability law on the subsets of (1, . . . , k) such that C{i, n) = {ji, ■ ■ ■ ,jp) with probability 
P*{(ji, . . . ,jp)}. We denote by Xi{n) the date of completion of task n at processor i, and by 
Aji{n) the duration of the synchronization constraint between nodes (i,n) and (j, n + 1) (it 
may include a transmission time as well as the execution time at processor j). We adopt the 
convention that Aji{n) = — oo if j C{i, n). It is easy to check that such a model, we could call 
it a task graph with random precedences, verifies Equation ([2]). 

A Queuing Network model studied by Baccelli & Liu [5] corresponds to this model. The task 
resource models studied in [14] or p2j also have this kind of structure. 

2.2 Cyclic Jackson network 

We consider a closed Jackson Network. The study of such closed networks can be traced back 
to Gordon and Newell, [25j. In their original model, there is a given number of indistinguishable 
customers. The routing of the customers leaving a given queue is provided by a sequence of i.i.d. 
Bernouilli random variables. All the service times are exponential. They prove the existence of 
an explicit product form for the unique stationary distribution. 

A natural generalization of the basic model is to consider i.i.d. (resp. stationary and er- 
godic) sequences of service times with general distributions, i.e. to replace ./M/l/oo servers 
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by ./GI /l/oo (resp. ./G/l/oo) servers. Finding the minimal assumptions leading to a unique 
stationary regime for this generalized closed Jackson Network is still an open problem. 

We consider a restriction of the previous model. There are k queues and all customers have 
the same cyclic route (1, 2, ... , k, 1), see Figured) We will denote this model by CJN for Cyclic 
Jackson Network, following the terminology of [27] . 



./GJ/l/oo FIFO ./G//1/00 FIFO 




Queue k 



Figure 1: A Cyclic Jackson Network consisting of k queues. 



In the following, the numbering of queues has to be understood modulo [A;], for example queue 
{k + 2) is queue 2. We denote by {(Tj(n),n € N}, the sequence of service times at queue j. 
Instead of describing the system by the workload or the queue length process, as is usually done, 
we propose to study this model by introducing the following variables. With each queue j, we 
associate a dater {xj{n)^n G N}. The variable Xj{n) represents the date of completion of the 
n— th service at queue j. All variables of interest for the network can be derived from these 
daters and from the sequences of service times. More precisely, we have: 



Asymptotic throughput at queue j: 



7j = lim 



n 



n >+oo Xj{n) 

• Idle time of queue j before the arrival of the n-th customer to visit queue j. 

Ij{n) = Xj{n) — (7j{n) — Xj{n — 1) . 

• Workload at queue j at the instant of the arrival of the n-th customer to visit queue j. 
This customer comes from queue j — 1. We suppose that it was the n'-th customer to visit 
queue j — 1. 

Wj{n) = Xj{n) — aj{n) — Xj^i{n') . 

The variables {■ji) which are obtained as ratios of daters will be called first order variables. The 
ones {Ij,Wj) which are obtained as differences of daters will be called second order variables. 
We are concerned with the problem of deriving necessary and sufficient conditions under which 
there is a unique stationary regime for both first and second order variables. In such a case, we 
say that our model is stable. 
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Suppose for the moment that there are exactly k customers. We suppose also that there is ini- 
tially one customer in each queue. These assumptions together with the FIFO service discipline 
at each queue yields the following property. The n-th customer to visit queue j will be, at the 
next step of its route, the (n + l)-th customer to visit queue j + 1. As a consequence, we have 



xi{n + 1) 
X2{n + 1) 

Xj{n + 1) 



max(j;i(n), Xfc(n)) + ai{n) 
max(a;2(n),xi(n)) +cJ2(n) 



max 



{xj{n),Xj-i{n)) + aj{n) 



Using the (max,+) notation, this can be rewritten as: 

/ o"i(n) — oo 

£72 (n) £72 (n) 

x{n + 1) = A{n) x{n), where A{n) 



\ 



-oo 



-oo 



-oo 



— oo 0"i(n) \ 

— oo 
— oo : 
— oo 



(5) 



The initial condition is x{0) ^ 0, where Xi(0) is the remaining service time of the customer being 
served at queue i at time 0. 

When the service times are deterministic, it is possible to obtain many asymptotic behaviors, 
depending on the initial condition x(0). In fact, initial delays between customers might never 
vanish. Therefore, it is possible to have several stable regimes for second order quantities 
(Ij, Wj, . . .) including periodic ones. For stochastic systems, when the service times are random 
variables, it is still possible to have several stationary regimes if the system is not "stochastic 
enough". As an application of the results presented in this paper, we obtain the necessary and 
sufficient conditions for the existence of a unique stationary regime for this CJN. This model 
will be used as an illustration of the results throughout the paper (Examples 14.81 16.91 16.161 17.7P . 



When there are less than k customers in the network, the system can be represented in the 
same way as previously. The only difference is that the structure of matrices {A(n)} is more 
complicated. When there are more than k customers, the trick consists in splitting queues. Each 
queue which has originally (p > 1) customers in its buffer is transformed into p queues with one 
customer per buffer. This is done by creating p — 1 fictive queues with service times identically 
equal to zero. By doing this, one gets back to the previous case. The main difference is that we 
have represented our model by a (max,-|-) linear system of dimension greater than the original 
number of queues. For more details on these transformations, see |30) . 

We can also model CJN with finite buffers (the ./G/l/L case). Finite buffers imply the blocking 
of some customers. There are different possible types of blocking. 

1. Blocking before service. The service begins at queue i only when the buffer at queue (^ + 1) 
is not full. 

2. Blocking after service (of communication type). Service at queue i begins as soon as a 
customer is available. After completion of the service, if the buffer of queue {i + 1) is full, 
the customer starts another service at queue i. 
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3. Blocking after service (of manufacturing type). Service at queue i begins as soon as a 
customer is available. After completion of the service, the customer has to wait in queue 
i if the buffer of queue {i + 1) is full. It prevents another customer from being served at 
queue i. 

4. Blocking after service (of Kanban type). The mechanism is the same as previously but 
there exists a finite intermediate buffer between queue i and (^+1). A customer completing 
its service at queue i enters this intermediate buffer if the buffer of queue (i + 1) is full 
and the intermediate buffer is not. It enables to serve a new customer at queue i. 

Excepting the blocking after service of communication type, all these types of blocking can be 
considered. We can even consider different types of blocking for the different queues of the 
network. In all cases, we obtain a (max,+) linear representation for the network. 

In the case of a CJN with i.i.d. general service times (./GI/1 servers), the classical method for 
studying the network is to consider the Markov chain formed by queue lengths and remaining 
service times and to apply Harris regeneration techniques. This method was first introduced 
for closed acyclic Jackson Networks by Borovkov [8], [9|. For closed Cyclic Jackson Networks, 
it is used by Bambos [6] and Kaspi & Mandelbaum [27] [28]. The method provides sufficient 
conditions of stability. Our approach is completely different and provides necessary and sufficient 
conditions of stability for CJN. 

For a much more complete description of the systems modeled by Equation ([2]), the reader is 
referred to the textbook of Baccelli, Cohen, Olsder and Quadrat [4J. 



3 (max,+) Algebra 

Definition 3.1 ((max,+) algebra). We consider the semiring (M U {— oo}, ©, (8i). The law © 
is "max" and © is the usual addition. We set £ = —oo and e = 0. The element e is neutral for 
the operation © and absorbing for (g). The element e is neutral for ^. The law © is idempotent, 
i.e. a® a = a. (M U {e}, ©, ®) is an idempotent semiring, called a dioid. It is usually referred 
to as the (max,+) algebra. We shall denote it by Mmax- 

We define the product spaces M^^^, M,^^^.. We define the product of a vector by a scalar: 
a G ^rnaxiU £ M^„^, (a © 'u)i = a © Uj. 

Matrix product is defined in a natural way, replacing + and x by © and © respectively. Let 
A,Be K^^^, 

{A © B)ij = max{Aii + Bi^) = © Bij . 

I 

Matrix- vector product is defined in a similar way, see Section [TJ In the rest of the paper, the 
notations "+,x" will stand for the usual addition and multiplication. Nevertheless, we will write 
ah for a © 6 whenever there is no possible confusion. For example, for A S Kmax) — ^'^'^ — 
A(g)-y^A . 

d factors 

Let us recall some definitions adapted to the Mmax algebra. 
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Definition 3.2. The graph of a square matrix A is a directed graph having a number of nodes 
equal to the size of A. This graph contains an arc from i to j iff Aji ^ e. The valuation of this 
arc is Aji. 

Definition 3.3. A square matrix A is irreducible if: yi,j 3n ^ | > e (or equivalently 

if its graph is strongly connected). 

Definition 3.4. A square matrix A is aperiodic if: 3N, Vn ^ N, yi,j, {A'^)ij > e. 

Definition 3.5. Let (i7,jF, P) be a probability space. A stochastic matrix {A{(jj), w G 1^} has 
a fixed structure if P{Aij = e) = 1 or P{Aij = e) = 0, Vi,j. 

Definition 3.6 (PE*^). The projective space FM.^ is defined as the quotient o/M'^ by the paral- 
lelism relation: 

n, u € u ^ V <^=^ 3a G R such that u = a ^ v . 
Let vr be the canonical projection ofM.^ into PM'^. 

For example (e, — 1)' and (2,1)' = (e + 2,-1 + 2)' are in the same parallehsm class, i.e. are 
two representatives of the same element of PM*^. We define in the same way PM^J^^^,, PM^ax ^-'^d 
p]^fcxA:_ ^YVe use the same notation vr for the different canonical projections. We define a norm 
and a distance on PM'^ which we call the projective norm and distance. 

Definition 3.7. Let x € PM*"' and u gM.^ be a representative of x, i.e. 7r(n) = x. We define: 

\x\p = maxuj — minuj . 

i i 

Let x,y G PM'^ and u,v &M.^ be two representatives of x and y respectively. We define: 
d{x, y) = d{u, v) = \x- y\'p = ^{ui - Vi) ^{vi - Ui) . 

i i 

We write either d{x, y) or d{u, v) with some abuse of notation. 

The space (PM'^, |.|-p) is an Euclidean space. In particular, it is complete. The norm 
corresponds to the Coo nor nfl on the projective space PR . 

Proposition 3.8. Let A G Rmax '^'^ irreducible matrix. Let u,v be two vectors ofM.^. We 
have: 

d{Au,Av) ^ d{u,v) . 

Proof. By definition, we have: 

d{Au,Av) = 0((An), - (Av),) ^{{Av)i - (Au),) . 

i i 

^It is worth mentioning that d{., .) is the M.max analogue of a distance used in classical Perron- Frobenius theory, 
which is called the Hilbert's projective metric and is defined by "S{u, v) — In (inf{/i/A | A?i ^ « ^ fiu})" . 
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We define such that {Au)i = 0^ Aij (g) Uj = ^ij(i) <8) Uj{i)- Note that depends on A and 
u. We have: 

^{{Au\-{Av),) = ^l(^A,,0u,)-{^A,^v,] 

i i \ 3 j 

« \ i 

j 

= © ^i(i) - ^ © - 

i i 

We obtain ^u) ^ 0j(ui — Wj) (g) 0j(vj - Uj) i.e. ^u) < d{u,v). □ 

There is no simple criterion to get a strict inequahty. This monotonicity has to be interpreted 
as a synchronization property. 

Definition 3.9. We consider A G MJ^^^. We set 

D{A) = sup d{Au, Av) . 

We call B{A) the projective diameter of A. 

It is easy to prove that D(yl) is finite if and only if Vi, j, Aij > e. A matrix A can be considered 
as a "linear" (in the (max,+) sense) operator from PM'^ into PM'=. It is a bounded operator if 
the (decreasing) sequence T){A^) has a finite limit, i.e. if A is aperiodic (Def. 13. 4p . 

4 Deterministic Spectral Theory 

We recall some results of the deterministic spectral theory in the ^max algebra. Theorem 14.21 
is due to Cuninghame-Green [18] . Versions of Theorem 14.41 were proved in [3l] [19] and [23] . 
Under the form proposed here, the result is from [16]. Theorem 14.51 is due to Cohen, Dubois, 
Quadrat and Viot [l5] and [E]. A complete treatment of the spectral theory can be found in 

in- 

We want to find non trivial solutions to the equation: 

A (g) X = A X , 

where ^ G R^^'^ is an irreducible matrix, a; is a column vector (the "eigenvector") and A is a 
real constant (the "eigenvalue"). We define also periodic regimes for the eigenvalue problem. 
A periodic regime of period d is a set of vectors {xi, . . . ,Xd} of M.^ verifying Axi = Axj+i, i = 
1, . . . , d — 1 and Axd = \xi. 
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Definition 4.1. For each path C, = {ti,i2, • • • itjitj+i = ti}, we define its average weight by: 

PiC) = atltj ® • • • ® atst2 at2ti/j , 
(here the division is the conventional one). 

Theorem 4.2. There is a unique (non e) eigenvalue, A. It satisfies the relation 

A = max p{C) , 

where ^ covers all the circuits of (the graph of) A. We call also A the Lyapunov exponent or the 
cycle time of A. 

There might be several eigenvectors. A hnear combination (in ^max) of eigenvectors is an 
eigenvector. An eigenvector has all its coordinates different from e (due to the irreducibility 
assumption). 

Definition 4.3. For a matrix A, we define: 

Critical circuit A circuit ( of A is said to be critical if its average weight is maximal, i.e. if 
piO = A. 

Critical graph It consists of the nodes and arcs of A belonging to the critical circuit(s). 
For a general graph, we define : 

Cyclicity The cyclicity of a strongly connected graph is the greatest common divisor of the 
lengths of all the circuits. The cyclicity of a connected graph is the least common multiple 
of the cyclicities of its maximal strongly connected subgraphs (s.c.s.). 

We normalize a matrix by subtracting (in the conventional algebra) the eigenvalue to all the 
coordinates. The eigenvalue of a normalized matrix is e. For a normalized matrix A of size k, 
we define: 

A'^ = A® A^ ® ■ ■ ■ ® a'' . 

We check that A+ © A''+'^ = A+. 

Theorem 4.4. Let A be a normalized matrix. 

a. Critical columns A'\, i belonging to the critical graph, are eigenvectors. 

b. For i,j belonging to the critical graph, tt{A'^) and Tr{A'j) are different iff they belong to 
two different s.c.s. of the critical graph. 

c. Every eigenvector of A writes as a linear combination (in ^max) of critical columns A^. 

In 

^maxi every irreducible matrix is cyclic in the sense of the following theorem. 
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Theorem 4.5. For an irreducible matrix A of size k and whose eigenvalue is X, there exists 
integers d and M such that: 

Vm ^ M, = A®'^ ® A"^ , (6) 

furthermore the smallest d verifying the property is equal to the cyclicity of the critical graph of 
A. We call it the cyclicity of A. 

A cyclicity greater than one will provide periodic regimes of period greater than one for the 
eigenvalue problem. 

Proposition 4.6. An irreducible matrix has a unique eigenvector and no periodic regimes of 
period greater than one for the eigenvalue problem, if and only if its critical graph has a unique 
s.c.s. and its cyclicity is one. Such a matrix will be called a scsl-cycl matrix. 

The proof follows from Theorems 14.41 and 14.51 

Definition 4.7 (rank). By analogy with classical linear algebra, we define the "rank" of a 
matrix A as the number of additively independent columns (resp. lines) of A. More precisely, 
let A i denote the i-th column of A. Matrix A is of rank r if there exists J C {1, . . . , fc} such 
that = r and Vi 7^ j € J', vr(y4.j) 7^ vr(A.j) and Vi J', 3aj,j € J^, such that 

TT{A,i) = 7r[0aj Aj] • 

Let ^ be a rank 1 matrix. Then A is a scsl-cycl matrix and verifies = X (E) A {X is the 
eigenvalue of A). The other way round, let A he a scsl-cycl matrix and M be defined as in 
Equation ([6]). One can check that A*^ is a matrix of rank 1. 

Example 4.8. [Cyclic Jackson Network 1] Let us consider a basic Cyclic Jackson Network as 
presented in Section [2^21 We suppose that the service times are deterministic, i.e aj{n) = aj. 
We suppose also that the number of customers, k, is equal to the number of queues. Then we 
can consider the (max,+) matrix associated with the network, see Equation ([5]). The graph 
associated with this matrix is constituted by the circuit (l,2,...,/c,l) and the recycling loops 
(1, 1) to {k, k). Let us define I = {i\ = maxj aj}. There are two possible cases. 

• If the cardinal |/| < k, then the critical graph of the matrix consists of the nodes i G I 
and the arcs {i,i),i G I. It implies that the matrix is scs|/|-cycl. 

• If |/| = k then the graph and the critical graph of the matrix coincide. It implies that the 
matrix is scsl-cycl. 

We conclude that the matrix is scsl-cycl if and only if \I\ = 1 or k. 
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5 Borovkov's Renovating Events Theory 

Borovkov's theory deals with the problem of regeneration in so-called "Stochastic Recursive 
Sequences". For a complete treatment, the reader is referred to Borovkov [7], Borovkov & Foss 
\10\ [TT] or Brandt, Franken & Lisek [13]. Let {^},J^,P) be a probability space. Let be a 
measurable map from JF) into itself such that P is ^-invariant and 0-ergodic. Let {E, £) and 
{G,Q) be two Polish spaces (complete, separable metric spaces) equipped with their respective 
Borel (T-algebra. 

Definition 5.1. We call Stochastic Recursive Sequence (SRS), a sequence {x(n)} of E -valued 
random variables defined by 

x{n + 1) = f{x{n), a(n)), n ^ 0, x{0) = xq , 

where {a(n)} is an exogenous sequence of G -valued random variables, stationary with respect to 
the shift 9. The function f is a measurable function from E x G into E. The vector xq £ E 
is the initial condition. In order to stress the value of the initial condition, we will sometimes 
denote the SRS by xq)}- We talk of an i.i.d. SRS when the sequence a{n) is i.i.d. (an 

i.i.d. SRS is a Markov Chain and the converse is true). 

Definition 5.2. We consider a SRS {a;(n)}. We denote by Ti the a -algebra Ti = a{a{n), 
n € {— oo, — 1}}. The sequence of events {A{n) € J^n+m, n € N} zs said to be a renovating 
sequence of length m and of associated function (p ■ — > E if: 

3no, Vn ^ no, x{n + m) = {a{n),a{n + 1), • ■ ■ , o(n + m — 1)) on A{n) . 

A sequence {A{n),n G N} of renovating events of same length and associated function is said to 
be stationary if A{n) = A{0) o 6^" = 6'-"^(0). 

We need the following notions of convergence: 

Definition 5.3. We say that there is coupling convergence infinite time (or, merely, coupling) 
of a sequence {X„} to a stationary sequence {Y 06*"} if 

PiXn+l = Yo 0"+', V/ ^ 0) 1 . 

It is easy to show that this notion of coupling convergence implies total variation convergence 
(Xn ^ y in total variation if sup^g^ G A) - P(Y G A)\ 0). 

Definition 5.4. We say that there is strong coupling convergence infinite time (or, merely, 
strong coupling) of a sequence {Xn} to a stationary sequence {Y o 9"^} if: 

V = min |n ^ 1 X^+i o 6'"("+') = y, V/ ^ o| is a.s finite . 
Remark Strong coupling implies coupling but the converse is not true. 
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Theorem 5.5 (Borovkov's renovating events). We consider a SRS {x{n)} defined by: 

x{n + 1) = f{x{n), a{n)), n ^ 0, x(0) = xq . 

If the random process {x{n),n G N} admits a stationary sequence of renovating events {A{n)} 
such that P(^(0)) > 0, then there exists a finite random variable Z such that: 

Zoe = f{Z,a{0)), 

and x{n) converges with strong coupling in finite time to Z o 9'"' . 

In the previous theorem, we have considered a SRS defined with a unique initial condition, xq. 
In the rest of the paper, we will be interested in having results that hold uniformly over the 
initial conditions. We will then use the following generalization of Borovkov's theorem. 

Theorem 5.6. We consider a subset V of E (V = E is in particular possible). We suppose 
that there exists a stationary sequence of events {A{n)} verifying P{A{Qi)) > and which is 
renovating for the SRS {x(n,xo)}, Vxq € V. Then, for all (possibly random) initial condition 
x(0) such that P(x(0) G y) = 1, the sequence {x{n)} converges with strong coupling to a unique 
stationary regime. 

Theorem 5.7 (converse of Th. I5.5l and l5.6p . The conditions of Theorem 1 5. 51 are necessary and 
sufficient for strong coupling convergence. Let V be a compact subset of E. The conditions of 
Theorem 15.61 are necessary and sufficient for strong coupling convergence uniformly over initial 
conditions in V . 

Next theorem was proved by Anantharam and Konstantopoulos in [1]. 

Theorem 5.8. Let {0,,J^,P) be a probability space. We assume that {i^,J-) is a Polish space 
equipped with its Borel a-algebra. We consider a SRS "x{n + 1) = f{x{n),a{n))" defined on 
E. Suppose that, for some xq G E, the sequence {x(n, xq)} is tigh^ on E. Then there is a 
stationary distribution for the SRS. 

The stationary distribution is defined on J7 x £" with an marginal equal to P. It provides only 
a weak stationary regime (wsr) for the SRS, see [1] or [13] for details. All we need to know about 
wsr is that stationary regimes are wsr. Hence, the uniqueness of stationary regimes implies the 
uniqueness of wsr. 

It is proved in [lOj, that for an i.i.d. SRS (i.e. Markov chain), the conditions of Th. 15.51 are 
equivalent to the ones ensuring Harris ergodicity. In Harris' framework, the conditions are on 
the state space. In Borovkov's framework, the conditions are on the exogenous driving sequence. 
This second approach is better suited for our problem. On the one hand, a direct analysis on 
the state space appears to be almost inextricable. On the other hand, the renovating events will 
take a very convenient form because a product of matrices is still a matrix (see Theorems 16.81 

EM- 

^Tightness on E means that for any 77 > 0, there is a compact K oi E such that P{x{n, xq) G K} > 1 — ry, for 
all n. 
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?7-coupling Coupling and strong coupling, introduced above, are related to total variation 
convergence. We define now the notion of ?7-coupling. It is related to weak convergence. 

Definition 5.9 (ry-coupling). We consider a metric space {E,d). We consider two sequences 
{Xn}neN o-nd {Yn}neN defined on E. We say that there is rj-couplin^ of these two sequences if 
for each r] > 0, one can find versions o/{X„} and {Yn} defined on a common probability space 
and an a.s. finite random time N such that 

n-^N ^d{Xn,Yn)^7]. 

The following proposition is shown in Asmussen [2]. 

Proposition 5.10. We consider a sequence {Xn]n&n and a stationary sequence {Y o 0"}„gp^ 
defined on the metric space E. Let fi be the invariant distribution of Y . If there is rj-coupling 
of the two sequences, then {Xn} converges weakly to /i. 

6 Presentation of the Results 

Let us consider a probability space {Q,J^,P,9). The probability P is stationary and ergodic 
with respect to the shift 6. We are interested in systems of the type: 

J x{n+l) = A{n)(^x{n), nSN 
1 x{0) = xo 

where x{n) and A{n) (Vn) are finite, respectively M^^^, and M^^^-valued, random variables. We 
are sometimes going to use the notation x{n, xq) to emphasize the value of the initial condition. 
We will consider models where the sequences {A(n),n £ N} are respectively i.i.d or stationary 
and ergodic (i.e A{n + 1) = A{n) o 9). 

We recall that we have defined first and second order limits in Section [H Equations ([3]) and (0]) . 
We are going to recall results on first and second order limits for open systems and first order 
limits for closed systems before completing the picture by solving the problem of second order 
limits for closed systems. 

6.1 Results from Baccelli [3] and Cohen |17| 

For X G M'^ and A G M^^^, we use the notation ||x||oo = 0(Li and ||^||oo = 0i^j=i ^jj- 
6.1.1 First order limits for closed systems 

Theorem 6.1 (Cohen [ITJ). Let {A{n)} be a stationary and ergodic sequence of matrices. We 
suppose that the matrix A(0) has a fixed structure (see Definition \3.5\) . is irreducible and verifies 

^The classical terminology is e-coupling. We change it to r;-coupling to avoid confusions with the notation 
e — ~QO of the Mmax algebra. 
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P{Aij(0) = e) = I or e < E{Aij{0)) < +00. There exists a constant A € M such that, for all 
initial condition xq and for all i G {1, . . . ,k}: 

n n n y n J 

The constant A is called the Lyapunov exponent of the stochastic matrix ^(0). 

The basic idea is to use the inequahty ||j4(g)i?||oo ^ H^Hoo^S) ||^||oo in order to apply Kingman 
subadditive ergo die theorem. 

This definition of a Lyapunov exponent is coherent with the one of Theorem 14.21 Indeed, 
by Theorem 14. 5^ for every irreducible and deterministic matrix A, there exists d and M such 
that Vm ^ M, ^'"+'^ = A*^ (g) A^, where A is the eigenvalue of ^4. It implies that Vxq G 
M^„^,hm„^"xo/n = A. 

6.1.2 First order limits for open systems 

We suppose that matrices A{n) have a fixed structure. We decompose the graph of ^4(0) into its 
maximal strongly connected subgraphs (s.c.s.). If we replace each s.c.s. by one node, we obtain 
an associated reduced graph which is acyclic. We associate with each node u of the reduced 
graph a constant Xu which is the Lyapunov exponent of the corresponding s.c.s. in isolation, see 
Theorem 16.11 We denote by 'u the set of predecessors of u (including u) in the reduced graph. 
We have : 

Theorem 6.2 (Baccelli ^). Let {A{n)} be a stationary and ergodic sequence of matrices. 
We suppose that A{0) has a fixed structure. We suppose also that P{Aij{0) = e) = 1 or 
£ < E{Aij{0)) < +00, Vi, j. Let us consider i S {1, . . . , k}, i belongs to the s.c.s. u. 

n n n \ n I 

Intuitively, the dynamic of the system is imposed by the s.c.s. having the smallest throughput 
(largest cycle time A). 

6.1.3 Second order limits for open systems 

Matrices A{n) have a fixed structure. In order to simplify the presentation of the results, let us 
assume that the structure consists of two s.c.s. The general case is completely similar. Up to a 
permutation of the coordinates, we have : 

^^""^ = ( ^(n) aU ) • 

The block U is a square matrix of size L x I, irreducible. It is interpreted as the input of our 
system. The block j4 is a square matrix of size (k — I) x (k — L) , irreducible. The block B is 
the matrix of the communications between the sources (U) and (A). We suppose that the block 
U in isolation has a unique stationary regime. We have the following theorem. 
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Theorem 6.3 (Baccelli [3]). Let u and a he the Lyapunov exponents of U and A respectively 
(see Theorem \6. Ifa<u, there is a unique stationary regime for the SRS Tr{x{n)), regardless 
of the initial condition. Convergence to the stationary regime occurs with strong coupling. If 
a > u, then the differences of the form 

Xj{n,xo) - Xi(n,xo), i = !,...,!, j = I + 1, . . . ,k , 

tend to +00, P — a.s., for all finite initial condition. 

If ti > a, the sources which are slower impose their pace. If it < a, everything happens asymp- 
totically as if A were in isolation. 

Remark In the previous theorem, we need the assumption that U in isolation has a unique 

stationary regime. Knowing if U has a unique stationary regime is precisely the problem which 
is going to be addressed in the following. Then, to determine if there is a unique stationary 
regime for 7r(x(n)), we have to use the results of Section [6.41 (applied to U) together with the 
comparison of Lyapunov exponents (of U and A) . 

Remark In the results above (Theorems 16. 11 [6^2] and [63|) . the assumption that a matrix M(0) 
has a fixed structure and is irreducible can be weakened and replaced by: 

limP ( M(n) ® M{n - 1) • • • ® M(0) irred. ) = 1, e < E{Mij{Q) \ Mij{0) ^ e) < +oo . 
6.2 Preliminary results 

Prom now on, we concentrate on second order limits in the closed (i.e A{n) is P-a.s. irreducible) 
case. The limits are expected to be random variables. We are interested in determining whether 
the limiting distribution is unique. Furthermore, we want to investigate the type of convergence 
to the limit. 

We recall that vr is the canonical projection -—)■ ¥M.^ (Def. 13. 6p . It is clear that the recursive 
equation x{n + 1) = A{n)x{n) defines a SRS (Def. 15. 1|) . It implies that 7r(x(n)) is also a SRS. 
Indeed, let us consider x{n) and x'{n) such that 7r(x(n)) = 7r(x'(n)). We define x{n + 1) = 
A(n)x{n) and x'{n + 1) = A{n)x'{n). It is straightforward that 7r{x{n + 1)) = Tr{x'{n + 1)). 
We write with some abuse of notation that {7r(x(n)), n G N} verifies the recursive equation 
"7r(x(n + 1)) = 7r^(n)7r(x(n))"|l|. 

Lemma 6.4 (Reising &: al [33j). For i S {1, . . . , k}, we define Zi{n) = Xi{n) — Xi{n — 1). We 
have Zi{n) = Fi{ A{n — l),7r(a;(n — 1)) ), where Fi is an absolutely continuous function. 

The sequence {j4(n)} being stationary by hypothesis, it implies the following corollary. 

Corollary 6.5. A sufficient condition for {zi{n),... ,Zk{n))' to converge weakly (resp. in total 
variation) to a unique invariant distribution, uniformly over initial conditions in PR'', is that 
7r{x{n)) has the same property. 

''it would be more rigorous to use different notations tt and tt for the canonical projections in PR'° and PR^^^ 
respectively. Then we would define more formally n{A{n))n{x{n)) = T7{A{n)x{n)). 
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This sufficient condition is not necessary as demonstrated by the following deterministic example. 
Example 6.6. Let us consider 




We have = A, so ui = (e, —1)' and U2 = (—1, e)' are eigenvectors of A. The set 

{ux = X(E)Ui e (1 - A) (g) U2, A G [0, 1]} , 

is the set of eigenvectors of A, see Theorem 14.41 There is a continuum of stationary regimes for 
7r(x(n)). For example, it is easy to check that for an initial condition ux, A G [0, 1], we have: 

xi(n, u\) — X2{n, Ux) = 2 x A — 1 . 

But on the other hand, we have a unique stationary regime for Zi[n). As a direct consequence 

of the equality A^ = A, we have zi{n) = Z2{n) = e, Vn ^ 2. 

We can also easily build stochastic counter-examples of the same kind. 

Remark The variables 7r(a;(n)) depend only on the sequence {7r(^(n))}. Therefore, all the 
results on 7r(x(n)) would still be true under the weaker assumption that only the sequence 
{7r(^(?7-))} is stationary and ergodic. But, on the other hand, the variables Zi[n) depend on 
the sequence {A(n)} and not only on {7r(^(n))}. Corollarv 16.51 would not be true under the 
assumption that {-7r(yl(n))} is stationary and ergodic. 

In the rest of the paper, we investigate the existence of a stationary regime for the SRS 7r(x(n)), 
i.e. the existence of a finite r.v. Z : O — > PR'^ such thatl 

Zo0 = 7r(^(O)^~^(Z)) . 

We are interested by conditions ensuring the uniqueness of the stationary regime and the con- 
vergence of 7r(x(n, xq)) toward it, for all xq G M'^. In such cases, we say that the model is stable. 
Two types of convergence will appear, convergence with 7/-coupling and convergence with cou- 
pling. They imply, respectively, weak convergence and total variation convergence as recalled in 

m 

6.3 Stability of discrete models 

Let {Ai, I ^ C OT I ^ N}, be a finite or countable collection of irreducible matrices of size k x 
k. We suppose that there exists a discrete probability law {pi} such that A{n,Lo) = Ai with 
probability > 0. 

Definition 6.7 (pattern, 1). 

A matrix A is called a pattern of the random sequence {A{n),n G N} if: 

1. 3N\A = Anjv-i (8) • • • O with no, ... , un-i G C (or NJ. 
^We will write Z o 6 — ■kA{Q)Z with some abuse of notations. 
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2. P{A{N - 1) • • • ® ^(0) = i) > 0. 



// the sequence {A{n)} is i.i.d. then the second condition is always verified. 

Theorem 6.8. The sequence of matrices {A(n)} is i.i.d. If there exists a pattern of {A{n)} 
whose critical graph has a unique s.c.s. and whose cyclicity is 1 ('scsl-cycl matrix), then 
{TT{x{n))} converges with strong coupling to a unique stationary regime. It implies total variation 
convergence of {7r(x(n))} to its stationary distribution. 

Proof. Let C = ^mjv-i • • • 'Si Auq be a scsl-cycl pattern. We have, using the cychcity 1 
assumption (Th. I4.5p . 

3M I Vm ^ M, C^^^ = AC"" , 

where A is the Lyapunov exponent of C. We conclude that for all initial condition y, C^^^^y = 
C{C^^y) = A (8> C^^y. It means that C^^y is an eigenvector of C. By the assumption on the 
critical graph of C, there is a unique eigenvector (up to a constant) denoted yo (Th. 14. 4p . We 
have C*^ <8> y = /i(y) S yo, fJ.{y) G or equivalently ■K{C^^y) = vr(yo)- We define 

Bi = A{i + MN -l,uj) '^■■■CdA{i + l,Lo)®A{i,uj) = C^'^} . 

From the i.i.d. assumption, it follows that ^{Bi) > 0. On Bi, and for all initial condition y, we 
have: 

x{i + MN) = C^^(^x{i) 
= fi{x{i)) (E) yo 
=^ 7r{x{i + MN)) = 7r(yo) • 

We check that the sequence Bi is compatible with the shift, i.e. Bi = Bo o 0'. We conclude 
that Bi is a stationary renovating event sequence for the SRS 7r(x(n)). We apply Borovkov's 
Theorem (version 15.61 for the set V = PM*^, as we have obtained a sequence of renovating events 
independent of the initial condition) and the uniqueness of the stationary regime follows. □ 

Example 6.9. [Cyclic Jackson Network 2] We consider a basic Cyclic Jackson Network with 
k queues and k customers. Such a network can be represented by the (max,+) matrix given in 
N2.21 Equation ([5|). We suppose that the sequence of service times {(c7i(n), . . . ,cjfc(n)) , n G N} 
is i.i.d. However the random variables (Ti(n), . . . , a"fc(n) need not be independent for a given n. 
We suppose also that the service times have a discrete support, i.e. can only take a countable 
number of values. We are in the framework of Theorem 16. 8[ We conclude that a sufficient 
condition of stability is to find a scsl-cycl matrix among the (max,+) matrices corresponding 
to this network. As a direct application of the result stated in Example 14.81 we obtain that a 
condition of stability is: 

P{3i I (Ti(n) > aj{n),\/j ^ i) > or P{ai{n) = 0"2(n) = • • • = (Tfc(n)) > . 



We now give a version of Theorem 16.81 in the stationary and ergodic case. 
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Theorem 6.10. The sequence {A{n)} is stationary and ergodic. We suppose that there exists 
a finite pattern C = Au^^^^ ® ■ ■ ■ ® Au^ which is scsl-cycl and of rank 1 (see Def. \4-T\ )- We 
suppose that B = {uj \ A{N — 1)^(A^ — 2) • • • A{1)A{0) = C} is of strictly positive probability. 
Then {7r(x(n))} converges with strong coupling to a unique stationary regime. 

Proof. The proof resembles the one of Theorem 16.81 As C is of rank 1, we have (see Def. 14. Th : 
= A (8) C, where A is the Lyapunov exponent of C. We conclude that: 

Vy G M^ C^y = C7(Cy) = XCy . 

It implies that Cy is an eigenvector of C. As matrix C is scsl, it has a unique eigenvector yo, 
up to a constant. On Bi = B o 0^, we have 

TT{x{i + N)) = 7r{Cx{i)) 
= 7r(yo) • 

We check that the sequence Bi is compatible with the shift and we apply Borovkov's Theorem 

E3 □ 

Remark If the dependence between matrices is markovian, a sufficient condition to get P{B) > 
is that p{Aui, Am^^) > 0, \/i = 1, . . . , N — 1, where p{., .) is the markovian transition kernel. 

Remark The conditions of this theorem are, of course, weaker than the i.i.d. assumption of 
Theorem 16.81 However we made an additional assumption, namely that the pattern C is of rank 
1. This assumption cannot be relaxed, as shown by the counter-example 16.111 

Example 6.11. Let = {uji,u>2} be the probability space, P={^, ^} the probability law, and 
6 the stationary and ergodic shift defined by: 0{uji) = uj2 and 9{uj2) = oJi- We consider 

■^-('7 '«^- 

{A{n,uji)} = A,B,A,B,... {A{n,uj2)} = B , A,B , A, . . . . 

Both matrices A and B are scsl-cycl patterns of length 1. But patterns which are scsl-cycl 
and of rank 1 are for example A^ or B^ for n > [1/r]]. We have for any n > [1/ry], P{3N \A{N — 
!)■■■ A{0) = = P{3N I A{N - 1) • • • ^(0) = B") = 0. Hence the conditions of TheoremETO] 
are not verified. In fact, there is a continuum of possible periodic limits. Consider xq = (a, b)' 
with —1 + r]<a — b<l — 7]. Then the limit regime of 7r(x(n)) has a state space which is either 
{7r(a, 6)', 7r(a + ??, 6)'} (with probability |), or {7r(a, 5)', 7r(a, 6 + r/)'} (with probability ^). 

6.4 Stability of general models 

In this section, we consider a general model where the coordinates of our matrices have a support 
which can be discrete, absolutely continuous with respect to Lebesgue measure or a mixture of 
these two cases. 



18 



We need the following definition, extending the notion of pattern we have been using for finite 
models. Let M be a deterministic matrix and ?? > 0. We denote by B(M,r]) the open ball of 
center M and of radius rj for the supremum norm of M'^^'^. We have € B{M, r]) iff 

Nij e]Mij-r],Mij+r][ 

Definition 6.12 (pattern, 2). Let A be a random matrix. We say that A is a pattern of A if 
A is a deterministic matrix verifying 

Vt? > 0, G B{A,r])^ > 0. 

Equivalently, we can say that A belongs to the support of the random matrix A. It includes the 
cases where A is an accumulation point (discrete case) or a boundary point (continuous case) of 
the support. 

Definition 6.13 (pattern, 3). Let {A[n),n £ N} be a sequence of random matrices. We say 
that the deterministic matrix A is a pattern of the sequence {A{n)} if 

3N s.t. Vr/ > 0, P ^A{N - 1) • • • ^(0) G B{A, ?])} > . 

Equivalently, we can say that A is a pattern (Def. \6.12) of the random matrix A{N — 1)®---® 
A(0). We say that A is an asymptotic pattern of {A{n)} if 

yr]>0, 3Nr, s.t. P ^Tr{A{N^ - I) ® ■ ■ ■ ® A{0)) G 7r(B(i, r/))} >0. 

Remark This definition is coherent with the one given in Definition 16.71 for a discrete model. 
Note that, for convenience reasons, asymptotic patterns are defined in the projective space 

Theorem 6.14. The matrices A{n) are i.i.d. (resp. stationary and ergodic). We suppose 
that there exists a matrix C which is a pattern of {A{n)} (see Def. \6.13\) and which is scsl- 
cycl (resp. of rank 1). Then the SRS {7r(x(n))} has a unique stationary regime {Z o ^"}. 
The convergence occurs with r]— coupling. It implies weak convergence of TT{x{n)) to its unique 
stationary distribution. 

Proof. We prove directly Theorem 17.41 a stronger version of the result. It is done in Appendix, 
M □ 

Theorem 6.15. The sequence of matrices {A{n)} is i.i.d. or stationary and ergodic. We 
assume that there exists a set C of matrices such that : 

1. VC C, C is a matrix of rank 1. 

2. yC e C, C is a pattern of {A{n)}. 

3. 3N\P {A{N - 1) • • • ^(0) G C) > 0. 
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Then {Tr{x{n))} converges with strong coupling to a unique stationary regime. 



The conditions of Theorem 16.151 are stronger than the ones of Theorem 16.141 as we require the 
patterns of rank 1 to be of positive probabihty. On the other hand, we obtain a stronger type 
of convergence. 

Proof. Let us define B = {uj \ A{N - 1)A{N - 2) • • • A{1)A{0) G C} and Bi = B o 9\ Using that 
the matrices C (z C are of rank 1, we obtain that, on the event Bi, 7r(x(i + N)) is independent of 
the value of 7r(x(i)). It imphes that {Bn,n G N} is a stationary sequence of renovating events. 
The result fohows. □ 

Remark Theorems 16.81 to 16.151 do not require any aperiodicity (Def. 13. 4p assumption on 
matrices A{n). However, the pattern C whose existence is essential in all of these theorems is 
aperiodic. The condition "scsl-cycl" implies aperiodicity. 

Example 6.16. [Cyclic Jackson Network 3] We consider the same i.i.d. model as in Example 
16. 9i However, the distributions of the service times are now general. We obtain, by using 
Theorems 16.141 and 16.151 the stability under the condition: 

The support of the random vector (cJi(n), . . . ,crjt(n)) contains at least one point such that: 



If the previous condition occurs with strictly positive probability, we obtain total variation 
convergence. Otherwise, we obtain weak convergence. Here is a case with only weak convergence. 
We consider an i.i.d. CJN with three queues and three customers. We assume that cti = (72 = 1 
and (73 is uniformly distributed over [0, 1]. 

7 Converse Theorems 

We are going to prove converses of Theorems 16.81 16.101 16.141 and I6.15[ We will consider succes- 
sively finite and general models of type: "x(n + 1) = A(n)x{n)" where the matrices are of size 
k X k and are P — a.s irreducible. We will, moreover, always suppose that there exists a pattern 
whose projective diameter (Def. 13. 9p is finite, i.e. 



It implies lim„ P{D( A{n) ■ ■ ■ A{0) ) < +00} = 1, see the proof of Lemma |9. 11 This condition is 
very weak. In the i.i.d. case, it is enough that there exists a pattern which is irreducible and 
aperiodic. We comment further on this condition in Section [8l 

7.1 Finite models in Q^^^ 

We consider a finite model: "x(n + 1) = A(n)x(n)", with A{n) G {Ai, I e C = {1, . . . ,L}}. We 
assume that the matrices are irreducible. We assume also that the matrices Ai,l G C, belong to 
Qmax! that their coordinates are rational. 



3i I ai{n) > aj{n),\lj / i or such that ai{n) = a2{n) 



ak{n) . 



3n I P {D( A{n) ■ ■ ■ A{0) ) < +00} > . 



(7) 
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Theorem 7.1. The sequence of matrices {A{n)} is i.i.d. or stationary and ergodic. When 
there is a unique stationary regime, convergence to this regime occurs with strong coupling. A 
necessary and sufficient condition for the model to have a unique stationary regime is that there 
exists a matrix C verifying 

1. C is a matrix of rank 1 (Def. \4- 7| j. 

2. C is a pattern of {A{n)} (Def. 

Proof. It is given in Appendix, ^9.2i □ 

Theorem 17. II is not true in general when the matrices Ai,l E £, belong to M^^^, see the following 
counter-example . 

Example 7.2. We consider the matrices 

where < 17,17' ^ 1 and rj, rj' are not co-rational, i.e. rj/vj' Q. 

Let u = (ui, U2)' £ IK^i we set iIj{u) = U2 — U1. We identify PM^ and M using the function ■0o7r~^. 
The matrices A and B are scsl-cycl. Their respective and unique eigenvectors are ■i/'C^i) = ~1 
and 'ip{e2) = 1- For a vector u = (ui,n2)' such that V'C^) ^ [— li 1]) we have 

'ip{Au) = max{ip{u) — r], —1), ip{Bu) = m.m.[tp[u) + 77', 1) . (8) 

We consider a Markov chain defined on the set V'^i"!) 1] C M^. The transition probabilities 
are 

• For u such that V'(^) — 1 + 1 — Au) = 1/2, p(n, i?n) = 1/2. 

• For u such that V'(^) S [1 — r/', 1], p(u,ylii) = 1. 

• For u such that V'(^) ^ ~1 + ^]) -B^i) = 1- 
The behaviour of the Markov chain is illustrated in Figure [2l 

B A B ^ 

-1-1 + 77 1-77' 1 pm2~m 

Figure 2: Markov chain il){X{n)) on M. 

Let ^(ra) be a realization of the Markov chain. It is easy to check that this Markov chain 
is aperiodic. Under the assumption 77/77' ^ Q, one can prove using classical arguments that 
the set {^{X{n)),n G N} is P-a.s. dense in [—1,1]. It implies that the Markov chain is 
i^-irreducible where u is the Lebesgue measure on 7/^~^[— 1,1]. Hence there exists a unique 
stationary distribution Q for the Markov chain. It verifies Q{A) > for all event A such that 
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I'iA) > 0. For a complete treatment of Markov chains on continuous state spaces, see Meyn &: 
Tweedie [31j. 

Let us consider a stationary realization X{n) of the Markov chain (i.e. Vn, P{X{n) G .} = Q{-))- 
We define 



As X{n) is stationary, it follows that {A(n)} is a stationary and ergodic sequence. 

Let us consider the stationary-ergodic finite model + 1) = A(n)x(n)" and x(0) = xq € M^. 
Note that {x(n)} is not a Markov chain anymore. 

Let us consider a pattern C = An-i • • • of {A{n)}, i.e. P{A{n-l) ■ ■ ■ A{0) = yl„„i • • • ^0} > 0. 
Let corresponding path for the Markov chain X{n), i.e. 

Xq G lp~'^] - 1, 1[, Xi= AqXq, ...,Xn = An-lXn-1, 

and P{X{n) = x„, . . . , X{1) = xi \ X{0) = xq} > . 

Let us denote by c and c the minimal distances between Xp,p ^ n and the extremal points of 

1]. We have c = minp<j„(V'(xp) + 1) and c = minp<c„(l - ip{xp)). 
It follows from ([8]) that 



where Im(^) = {Au, u G M'^}. From the definition of the Markov chain X{n), it follows that 
c > 0,c > 0. We conclude that An-i • • • ^0 is not a rank 1 matrix. There exists no finite pattern 
of rank 1 for 

On the other hand, let us prove that there exists asymptotic patterns of rank 1 for {^(n)}. 
We define c(n) = va.inp^ri{'4'{X{p)) + 1) and c(n) = minp<g„(l — il){X{p))). As {V'(^(n)),n € 
N} is dense in [—1,1], we obtain that c(n) — > and c(n) — > 0. Using ([9j), we obtain that 
T){A{n) ■ ■ ■ A{0)) — > 0, P-a.s. We conclude following the lines of Theorem 17.41 ^9.3i There is a 
unique stationary regime for the model. For an arbitrary initial condition, we have //-coupling 
(weak convergence) with this stationary regime. 

To summarize, we have exhibited a finite model with a unique stationary regime and no coupling 
convergence. This type of behaviour is closely related to the non-finiteness of the projective 
semigroup n < A, B >, see Def. 19.21 

7.2 General models 

We consider a general model of type "x(n + 1) = A{n)x{n)" . Stability no longer implies coupling 
in finite time. It was illustrated by Example 17.21 Here is another example, for an i.i.d. model. 

Example 7.3. 



where U{n) are i.i.d. random variables of uniform distribution over [0,1]. There is a unique 
stationary regime for 7r(x(n)) which is 7r(e, e)'. We denote by d{., .) the projective distance. For 




A ii X{n + l,Lo) = AX{n,uj), 
B ii X{n + l,u:) = BX{n,u). 



■0(Im(A„_i • • • Aq)) = [Xn - c, x„ + c] 



(9) 
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an initial condition (y, e)' with y ^ 1, we have d {x{n); (e, e)') = minp^„ U{p). Thus convergence 
to 7r(e, e)' occurs only asymptotically. There is no coupling but only r/-coupling with the unique 
stationary regime. 

We can show the following results. 

Theorem 7.4. The sequence of matrices {A{n)} is i.i.d. or stationary and ergodic. The nec- 
essary and sufficient condition for the model to converge with ij- coupling to a unique stationary 
regime is the existence of an asymptotic pattern C of {A{n)} of rank 1 (Def. \6.13\) . 

Proof. It is given in Appendix, ^9.3[ □ 

Theorem 7.5. The sequence of matrices {A{n)} is i.i.d. or stationary and ergodic. The 
necessary and sufficient conditions for the model to converge with coupling to a unique stationary 
regime are : 

There exists a set C of matrices such that : 

1. VC C, C is a matrix of rank 1. 

2. yC e C, C is a pattern of {A{n)}. 

3. 3N\P {A{N - 1) • • • ^(0) G C) > 0. 

We can say equivalently that we must have patterns of rank 1 hut with strictly positive probability. 



Proof. We have already proved the sufficient part (Th. I6.15p . We prove the necessary part of 
the theorem in Appendix, §9.4[ □ 

Convergence with r/— coupling appears as a limiting case of coupling in finite time. In a discrete 
model, we will have only ry-coupling when the set C of scsl-cycl patterns is non empty but is of 
probability 0. It means that the scsl-cycl patterns are only accumulation points of the support. 
In a general model, we will have only r/-coupling when the scsl-cycl patterns are isolated points 
of the support (which implies that they are boundary points of the support). 

Example 7.6. To illustrate the previous remark, let us continue the analysis of Example 17. 3[ 
There is only one matrix (in the projective space PM^^^) verifying the first two conditions of 
Theorem 17.51 It is the matrix 




But condition 3. of Th. [73] is not verified as V7V, P {tt{A{N - 1) • • • ^(0)) = tt{C)) = 0. 

Let us consider a slightly modified sequence of matrices {A(n)} where the diagonal elements are 

two random variables U{n) and U'{n) defined on [0, 1] and such that 

P{U{n) > U'{n)} > or P{U'{n) > U{n)} > 0. 

Now, we have scsl-cycl patterns with strictly positive probability and there is coupling in finite 
time with the unique stationary regime. 
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Example 7.7. [Cyclic Jackson Network 4] We consider the model of Example 16.161 The 
condition 

P {{3i I a,(n) > aj{n),\/j ^ i} U {ai{n) = ■■■ = ak{n)}) > . 

is necessary and sufficient for strong coupling convergence to a unique stationary regime. For 
i.i.d. Cyclic Jackson Networks, the sufficient condition P {{3i \ ai{n) > aj{n),\/j 7^ z}) > was 
obtained in [27]. The method of proof was completely different, see the remarks at the end of 
Section 

8 Without Irreducibility 

We have supposed from the beginning that the matrices {j4(n)} were irreducibles. The relaxation 
of the irreducibility assumption is very important in terms of modeling power. It enables us to 
consider, for example, the task graphs with random precedences introduced in Section 12. 1[ 

The irreducibility assumption is used in Prop. 13.81 But the only point we need to prove this 
proposition is: "Vn G M'^, Au G M*^", i.e. if u has only non-e coordinates then Au has the same 
property. So the only assumption we need on the matrices {j4(n)} is: 

I V«, P{3j s.t. Aij(O) > e} = 1 . 

The irreducibility is also essential for the Mmax spectral theory of Section HI A reducible matrix 
A G R*"'^*^ may have several eigenvalues. Definition 14.31 and Theorem 14 .41 have to be reinterpreted 
by replacing the unique eigenvalue by the maximal eigenvalue. Theorem 14. 51 is not true anymore. 
But as far as the direct theorems (16.81 16.101 16.141 and 16.150 are concerned, we use results of the 
I^max spectral theory only for the pattern C whose existence is critical for the proofs. These 
theorems are still valid if we state that condition I is verified and that there exists a pattern C 
which is scsl-cycl and irreducible. 

Dropping the irreducibility assumption does not infiuence the converse results. More precisely, 
the proofs of Theorems 17. H 17.41 and 17.51 are still valid. Only two conditions need to be verified: 

I Vi, P{3j s.t. ^ij(O) > e} = 1 . 

II 3n I P{D(^(n) • • -^(0) ) < +00} > 0. 

Irreducibility P — a.s. is not necessary to ensure that these conditions hold. We conclude that 
we can state our converse results under the previous two minimal assumptions. 

A counter-example shows that without condition II, the uniqueness of the stationary regime 
does not imply the existence of a rank 1 pattern. 

Example 8.1. Let Vt = {1^1,(^2} be the probability space, P = {^, ^} the probability law, and 

9 the ergodic shift defined by: 6{ui) = 012 and 6 {0)2) = oJi- We consider 




{A{n,LOi)} = A,B,A,B,... {A{n,u:2)] = B , A,B , A, . . . . 
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All patterns have an infinite projective diameter. Therefore, condition II is not verified. Nev- 
ertheless, there is coupling in finite time with a unique periodic regime. More precisely, there 
is coupling of 7r(x(n, n)) to the periodic regime {7r(e, e)', 7r(e, 1)'} and coupling occurs for n > 
2 X [0(tii — U2,U2 — ui)]. We conclude that there is coupling in finite time to a unique stationary 
regime but no rank 1 pattern. Without condition II, Theorem 17. II is not true anymore. 

Another class of systems where condition II is not verified is the class of open systems studied 
by Baccelli in [3]. The results for this type of systems have been recalled in §6.11 In this case 
also. Theorem 17.11 fails to be true. In such models, there are no patterns which are scsl-cycl 
and irreducible (matrices are non- irreducible with probability 1 !). The good criterion to decide 
the uniqueness of the stationary regime is the comparison between Lyapunov exponents, see 
Theorem 16.31 The computation of such exponents involves the whole structure of the stochastic 
matrices {A(n)}, and not only an extracted pattern. 

Condition II is weak and will be verified in most cases. For a discrete i.i.d. model, for example, it 
is sufficient to have one pattern of finite length A^j^ ■ ■ ■ A^-y which is irreducible and aperiodic to 
verify it. For a general i.i.d. model, it is sufficient to have P { A(0) irreducible and aperiodic} > 
0. In a stationary and ergodic framework, condition II is a little bit stronger, as shown by 
Example 18.11 where P{A(0) irreducible and aperiodic} = 1 and where condition II is not 
verified. 

Remark For a general model which does not verify condition II, we decompose the model into 
its maximal sub-models verifying it. Then the complete analysis of the system boils down to 
an analysis of the sub-models (using the results of Section 16. 4p and of their interactions (using 
Theorem 16.31 and its generalizations, see [3]). 

9 Appendix 

9.1 Loynes scheme 

Lemma 19.11 is going to be used in several of the forthcoming proofs. Under an assumption of 
7/-coupling of the trajectories, we build a stationary regime using a Loynes' type construction. 

Lemma 9.1. We consider a general model "x(n + 1) = A{n)x{n)" (see §g.^[ j. The sequence 
{A{n)} is stationary and ergodic. We assume that there exists N such that 

P {D( A{N) • • • A(0) ) < +00} > 

We assume also that \/xo,yo G M'^, d{x{n,xo),x{n,yo)) —fO,P a.s. (r]-coupling of the trajecto- 
ries). Then there exists a r.v. Z : Q ^ PM'^ verifying Z 06 = 7r(A(0))Z. The sequence {Zo^"} 
is the unique stationary regime of the model. 

Proof. We are going to show that the sequence {7r(^(— 1) • • • A{—n)u), n G N}, u € M'^, has a 
simple limit in PM'^. The argument is an analog of the famous backward scheme proposed by 
Loynes in [29] for G/G/1 queues. 

We want to show that D( A{n) ■ ■ ■ A{0) ) ^ 0, P — a.s. It is easy to see that the event 

A = {uj\ 3N, D( A{N) ■ ■ ■ A{0) ) < +c3o} = {lo \ 3N, Vn ^ N, D( A{n) ■ ■ ■ A{0) ) < +00} 
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is invariant by the translation shift. Then by the ergodic Lemma, it is of probabihty or 1. We 
have made the assumption that P{A) > 0, hence P{A) = 1. 

Using the stationarity of the sequence {A{n),n G Z}, we have that 3N{lo) such that 

D{A{-1)--- A{-N)) < +00. 

Then we can define the projective image of A{—1) ■ ■ ■ A{—N) which is a bounded subset of PM*"' 
and that we denote by 11. The boundedness imphes that 

c = max d(e, v) < +oo , 

where e = (e, . . . , e)'. Let us define the vectors 

ci = (c, e, . . . , e)', C2 = (e, c, e, . . . , e)' , . . . , Cfe = (e, . . . , e, c)' . (10) 

It is immediate that 11 is included in the convex hull of these vectors, i.e. 

n C {7r(ai ® ci e 02 «) C2 • • • © Ofc (g) Cfc), € M} . 

In the (max,+) algebra, we have the following property, for all A G RJ^^^, u, v G ^maxi A{u(Bv) = 
Au © Av. It implies 

Vx G n, 7r{Ax) G {^{ai (g) vlci © • • • © © Ack), G M} . (11) 

We fix 7] > 0. Using the ??— coupling assumption, we have that the random variable N'(uj) is 
P — a.s. finite, where A^' is defined by: 

A^' = inf{n | d{x{n, Ci),x{n, Cj)) ^ r], Vi, j} . 

As both N and N' are P — a.s. finite, we have 

V(5 > 0, 3L, L' : P{N ^ L, N' ^ L'} ^ 1 - 5 . 

As a direct consequence of (jlip . we have on the event {N ^ L,N' ^ L'} : 

D {A{L' - 1) © ■ ■ ■ © A{0)A{-1) © • • • © A{-L)) ^ 7] . 

We deduce, using the stationarity of {A{n)}, that 

P{d(^(-1)---^(-L-L')) < ??} = P {d{A{L' -I)--- A{-L)) s^T]] 

^ 1-6. 

It implies that the random variables X){A{—1) ■ ■ ■ A{—n)) converge in probability to 0. But as 
D(j4(— 1) • • • A{—n)) is pathwise decreasing, the convergence occurs also P — a.s. 

We have in particular, for ah n G M^, d ( A{-1) ■ ■ ■ A{-n)u, A{-1) ■ ■ ■ A{-n - p)u) — > 0, P - 
a.s. It implies that {7T{A{—l)---A{—n)u)} is a Cauchy sequence which converges. The limit 
does not depend on u. We denote it by Z. We have : 
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Zoe = lim7r(A(0)A(-l)--- A(-n)u) 

n 

= irAiO) lim7r(A(-l) • • • A{-n)u) = 7r{A{0))Z . 

n 

The sequence {Z o 0"} is a stationary regime. Let us prove it is the unique one. We want to 
prove that 

VxoGM^ d(x(n,xo),Zor)''^°°0, P-a.s. (12) 

As Z is P—a.s. finite, for all i] > 0, there exists a compact K € PM'^ such that P{Z € K} > 1— ry. 
We proceed as above (Equation (fTOll ) in order to define vectors Ci,i = 1, . . . , k such that 

K C {7r(ai ci © 02 «) C2 • • • © Ofc Cfc), G M} . 

We have 

Z£K^Zo9P£ {TT{ai[A{p -!)••• A(0)ci] © • • • © ak[A{p -!)••• A(0)cfc]), cti G M} . 

Using the r/— coupling of trajectories, we also have 

Vcj, d(x(n, xo),x{n, Cj)) ^ . 

We conclude easily that there is 77— coupling of {7rx(n, xo)} and {Z o 0"}. We can apply Propo- 
sition [5?Tni There is weak convergence of {7rx(n,xo)} to the distribution of Z and relation (jl2p 
establishes the a.s. convergence of {7rx(n, xq)} to Z o 0'"-. As a direct consequence, {Z o 9^} is 
the unique stationary regime. □ 

9.2 Proof of Theorem [HI] 

We are going to prove that the existence of a unique stationary regime implies the existence of 
a pattern of rank 1 (Def. 16. 7p . Using Theorem 16.101 the proof will then be complete. 
We need the following definition. 

Definition 9.2. Let us consider Ai, . . . , Ap G I^max- denote by < Ai, . . . Ap >, the semi- 
group generated by these matrices and by tt < Ai, . . . Ap > its projection. We have 

< Ai,...,Ap> = {{Au„- ■ ■ Au^Aui) , ui, . . . ,Un e {I,. . . ,p},n ef^} , 
7r< Ai,...,Ap> = {Tr{Au„- ■ ■ Au^Au-^) , ui,... ,Un e {I,... ,p},n en} . 

We consider the Euclidean space (PM^^'^, as introduced in Definition 13.71 Next proposition 
was proved by Gaubert in [21]. 

Proposition 9.3. Let Ai, . . . ,Ap G Q'^Jx- ^or all compact set K of (PM^^^, |.|p), we have 
TT < Ai, . . . , Ap > nK is finite. 

Let us prove a lemma first. 
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Lemma 9.4. We consider a finite model "x{n + 1) = A{n)x{n)" with A{n) € {Ai, . . . , Ap} 
and Ai G Qmax)^ — li • • • )P- ^6 suppose that there is a unique stationary regime. It implies 

d{A{n)A{n -!)••• ^(0)xo, A{n)A{n -!)••• ^(0)yo) 0, P - a.s . (13) 

Equivalently, it implies rj-coupling of the trajectories corresponding to different initial conditions. 

Proof. We assume that Equation ()13p is not verified. It implies, using Proposition 13.81 that 
there exists xq, yo S M'^ and c > such that 

PiA) > 0, A = {\imd{A{n)A{n-l)---A{0)xo,A{n)A{n-l)---A{0)yo) > c} > . (14) 

Let S =< Ai, . . . ,Ap > be the projective semigroup generated by the matrices of the model 
(Def. ESD- For x G M'^, we define S{x) = {tt{Ax), A £ S}. We have that tt < Ai, . . . , Ap > nK 
is finite for all compact K of PM'^'^'^, Proposition 19.31 It implies that S{x) PI is finite for all 
compact K of PM*^. We conclude that S{x) has no accumulation point and verifies S{x) = S{x), 
where S{x) is the closure of S{x) in PM'^. 

We want to apply Theorem 15.81 It is required that the probability space be a Polish space. In 
order to fulfill this, we consider the canonical probability space consisting of one-sided infinite 
sequences of matrices {Ai, . . . Ap}, i.e. 

^ = {{Auo,Aui,- ■ • ,^n„, • ■■ ), G {1, . . . ,p}} . 
We recall that we made the assumption ([7]), which implies 

lim P {D( A{n) ■ ■ ■ A{0) ) < +00} = 1 . 

n 

It implies that for all rj > 0, there exists N and K, a compact set of PM'^^'^ such that 

yn^^N, P {tt{A{N) ■ ■ ■ A{0)) e K} > 1 - rj . 

There exists a compact K' (which depends on x) of PM'= such that 

{tt{A{N) ■ ■ ■ A{0)) eK}^ U{A{N) ■ ■ ■ A{0)x) G K'} . 

We conclude that the sequence {7r(x(n,x))} is tight in PM'^. It implies that it is tight in S{x) = 
S{x). We can view 7r(x(n,x)) as a SRS defined on S{x) only. Applying Theorem 15.81 we obtain 
that, for all x, there exists a stationary distribution Qx defined on J7 x S{x). 

Let us consider the initial conditions xq and yo as defined in (jl4p . It is a-priori possible to have 
>S'(xo) n S{yo) 7^ 0. As a consequence, one cannot rule out that Qxq = Qyo- We are going to 
prove that there exists a G M such that S{xo) n S{axo yo) = 0. It will provide two stationary 
distributions Qxq 7^ Qaxoeiyo 1 which contradicts the uniqueness of the stationary regime. 

We work on the event A, see (fH|) . We have d{x{n,xo),x{n,yo)) > c for all n. Let G M'^ be 
two different points. Then there exists an open interval ]A, A[ such that 

\ - X = d{x,y), {Xx®y,Xx®y} = {x,y} , 
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VA 7^ A' e [A, A], Ax © y 7^ A'x y . 

The proof is straightforward. We consider the (random) intervals ]A(n), A(n)[ defined as above 
for the couples of points {x{n, xq), x(n, yo)}- For any A G M^^^, x, y G ^max ^iid A € M, we have 
A{Xx © y) = A^x © Ay. As a consequence, the sequence ]A(n), A(n)[ is decreasing. Let A and A 
be the limits of \{n) and A(n). On the event A, we have A — A > c (see p^ ). 

We define the sets 

A(n) = {A I 7r(Ax(n, xq) © x(n, yo)) G 5(xo)}, A = |J A(n) . 

neN 

Let x,y, z G PM'^ be three different points. It is immediate to prove that there exists a unique 
A G M such that z = Ax © y. As a consequence, the sets A(n) are countable and A is countable. 
It implies that the set ]A, A[\A is non-empty on A. For all A g]A, A[\A, we have, by definition of 
A, that ^(Axo © yo) H 'S'(xo) = 0. The conclusion follows. □ 

Remark The proof does not work when matrices Ai, . . . , Ap belong to MmaS- this case, it is 
possible to have S{x) ^ S{x). In the model detailed in Example 17.21 all the sets S{x) are dense 
in the interval [—1, 1] (as a classical consequence of the assumption rj/if Q). It implies that 

S{x) = [—1, 1], Vx. The stationary distributions Qx are all defined on the same set, x [—1, 1], 
which prevents the previous proof from working. 

We want to prove the existence of a rank 1 pattern of {^(n)} (Def. 16. 7p . There exists a r.v. N 
such that A{N) ■ ■ ■ A{0)ij > e,yi,j (consequence of Equation ([7])). 
It follows from the ergodic Lemma, that the set 

I={n\ n^N, A{n) ■ ■ ■ A{n - N) = A{N) ■ ■ ■ A(0)} (15) 

is infinite, P-a.s. Let a : N — > N be the strictly increasing function such that I = {a"(0), (t(1), . . . }. 
We define the subsequence {B{n) = A{a{n))A{a{n) — 1) • • -^4(0), n G N}. The matrices B{n) 
can be written under the form B{n) = A{N) ■ ■ ■ A{0)B{n)A{N) ■ ■ ■ A{0) for n ^ 3. We have 

maxB{n)ij < max A{N) ■■■ A{0)ij (g) max B{n)ij ^ max A{N) ■■■ A{0)ij 

ij ij ij ij 

^ max A{N)--- A{0)ij B{n)uv0^ay^A{N)--- A{0)ij , (16) 

for some indices u,v belonging to the argmax in maxjj B{n)ij. We also have 

Vi, j, B{n)ij ^ A[N) ■ ■ ■ A{0)iu © B{n)^^ © ^(iV) • • • ^(0)^^ 
mm B{n)ij ^ mm A{N) ■■■ A{0)ij B{n)u^ mm A{N) ■■■ A{0)ij . (17) 

ij ij ij 

We consider the Euclidean space (PM'=^*=,|.|p) where |.|-p is the norm introduced in Definition 
\3l\ It follows from ([16]) and that 

\B{n)\v = max B{n)ij - mm B{n)ij ^ 2 x {max A{N) ■ ■ ■ A{0)ij - min^(iV) • ■ ■ A{0)ij) 

ij ij ij ij 

= 2 X \A{N) ■ ■ ■ A{0)\v . 
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It implies that the sequence {7r{B{n))} belongs to a compact of (PM^^^, |.|-p). Hence there 
exists a strictly increasing function cr : N — > N such that Tr{B{a{n))) is converging. Let Aoo be 
a representative (in M'^^'^) of the limit. By continuity of the projective distance, we have that 
yu,v G M'^, d{AooU,Aoov) = 0. Therefore A oo is a rank 1 matrix. 

As the products {TT{A{n) ■ ■ ■ A{0))} can only take a finite number of values in compact sets 
(Proposition 19. 3( ). it implies that the limit matrix A^o is attained in finite time. More precisely, 
there exists such that 

Vn ^ N, TT{B{a{n))) =7r{A^) . 
The matrix B{a{N)) is a rank 1 pattern for {A{n)}. It concludes the proof. h 

9.3 Proof of Theorem ITTH 

We first prove the necessary part of the Theorem, i.e. ry— coupling with a unique stationary 

regime implies the existence of an asymptotic pattern. 

Let Z o 6*" be the unique stationary regime. We have for all xo,yo ^ , 

d{x{n, xq), Z o 9'^) ^ 0, d{x{n,yQ), Z o 0") ^ =^ d{x{n,xo),x{n,yQ)) ^ . 

We have assumed that 3N such that P{D{A{N -!)••• A{0)) < +00} > 0, see Equation ([ID, 
Section Let AT G R be such that P{D(^(-1) • • • A{-N)) < K} > 0. It implies that there 
exists K' such that P{\A{-1) ■ ■ ■ A{-N)\-p < K'} > 0. Let us denote 

£o = {uj\ \A{-l)---A{-N)\v<K'}. 

It follows from the stationary-ergodic assumption, that there exists a minimal ni > 1 such that 

P{£i} > 0, ^1 = ^'o n {\A{-ni) ■ ■ ■ A{-ni - N + l)\v < K'} . 

We define in the same way an increasing sequence Up > ■ ■ ■ > n2 > ni and a decreasing sequence 
of events C • • • C f 2 C £^1 verifying 

P{£p} >0, £p = £p-i n {\A{-np) ■ ■ ■ A{-np -N + l)\v<K'}. 

On the event £'p,p ^ 1, we have 

l^(-l) • • • A{-np - N+l)\p <2x K' . 

The proof is exactly similar to the one proposed in the proof of Theorem 17.11 ( ^9.21 Equation 
(fT5l) and after). Let B{E,K') denote the open bah of (PM'^^*^, \.\v) of center 7r{E),Eij = e,\/i,j 
and of radius K' . For all p, we choose a deterministic matrix Bp belonging to B{E,K') and 
verifying 

pS^{£p}n{7TA{-l)---A{-np-N + l)eB{Bp,^)}^ >0. (18) 

As the matrices {Bp,p e N} belong to a compact, there exists a subsequence {^^(p)} which 
converges to a limit i?oo- We have (see the proof of Lemma l9.ip that D(^(— 1) • • ■ A{—n)) 
0, P — a.s.. We conclude that Sqo is a rank 1 matrix. 
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We fix r/ > 0. Let C be such that Vp > C, we have \Boo — B^(p^\-p ^ r]/2. For p > max(C, 2/r]), 
we have B{B^(^p^,l/p) C B{Boo,r]). It imphes 

. . . A(-n,(p) - AT + 1) G B{B^,7])} ^ 

P n {7ryl(-l) • • • A{-n^(^p) -N + l)e > . 

It means precisely that B^o is an asymptotic pattern of {A{n)}, see Definition 16.131 

Let us prove the sufficient part of the theorem. We assume that there exists a deterministic 
matrix A which is a rank 1 asymptotic pattern of {A{n)}. We want to prove the r^-coupling 
convergence of 7r(x(n)) to a unique stationary regime. 
We fix ry > 0. Let A''^ be such that 

P {^wA{Nrj -!)■■■ A{0) G B{A, v)] > . (19) 

Using the ergodic Lemma, we have 

P{3i ^ I TTA{Nr, - 1 + i) • • • A{i) G B{A, 7])} = !. 

Let u be the unique eigenvector of the rank 1 matrix A and B{u, ij) the ball of center it{u) and 
radius r] in PR'^. We have that for all xq 

{7r(x(n, xo)) G B{u, ??)} C {7r^(n - 1) ■ ■ ■ A{n - iV^) G B{A, t])} . 

In particular, it implies that Vxo,yo £ ^-nd n large enough, 

{d(x(n,xo),x(n,yo)) < ??} C J {7ryl(p - 1) • • • - iV^) G ?])} . 

We deduce that xq), a;(n, yo)) < ?]} ^ 1- We conclude by using Lemma l9.ll (the 

existence of n such that P{'D(A{n) • • • A{0) < +00} > comes from Equation (fT9]) ). 



9.4 Proof of Theorem 17.51 

We want to prove that the conditions given in Theorem 17.51 are necessary. We suppose that our 
model couples in finite time with a unique stationary regime, uniformly over initial conditions 
in M'^. Let us prove a lemma first. 

Lemma 9.5. // there is a unique stationary regime for Tr{x{n)), coupling in finite time uni- 
formly over initial conditions in M*^ implies strong coupling in finite time uniformly over initial 
conditions in M.^ . 
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Proof. Let {Z o be the unique stationary regime with which the SRS 7r(x(n)) couples. We 
consider the event: 

y^ = ^uj\ TT{x{n, Xo)) o e-'^uj = Zu, Vxo E M'^j . 
The assumption of coupUng in finite time, uniformly over M.^, may be written : 

p{yn) 1 . 

Here we implicitly use the assumption that the projective image of A{—1) ■ ■ ■ A{—n) is asymp- 
totically bounded (see Equation ([T])). Let us consider w € 3^n and p an integer > 0, we have: 

7T{x{n + p,xo))o9-''-Puj = TT{x{n,x{p,xo)oe-P)) oQ-^'u (20) 

= Zuo (as UJ eyn) . (21) 



The passage from (|20p to (j2ip uses the fact that coupling occurs uniformly over initial conditions. 
We have: 

y^ = i^Lo\ 7r(x(n + p, xo)) o = Zlo, Vp ^ 0, Vxq E M'^} , 

and 

p{yn) 1 . 

This is exactly the definition of strong coupling (Def. 15. 4p . □ 

We can now use the converse Theorem 15. 71 There exists a stationary sequence of events {A o 
which is renovating for the SRS {vr ( x(n, xg) )}, Vxq € M*"', and verifies P{A) > 0. Let m be the 
common length and ^ the common function of these renovating events. We have, on A: 

7r(x(m)) = $ {A{m - 1), ... , A{0)) , Vxq G M'' . 

But we also have: 

x(m) = A{m - 1) (g) • • • A{0) (g) xq, Vxq G M'' . 

We conclude that, on A, vr (A(m — 1, w) (g • • • (g) A(0,a;) (g xq) is independent of xq. It implies 
that C = A{m — 1, u) (g • • • (g A(0, oo) is a matrix of rank 1. H 
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